Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Add support for exasol datasource #17166

Merged
merged 42 commits into from
Oct 31, 2024
Merged

feat: Add support for exasol datasource #17166

merged 42 commits into from
Oct 31, 2024

Conversation

Nicoretti
Copy link
Contributor

@Nicoretti Nicoretti commented Jul 24, 2024 •
edited
Loading

Hi @ all 👋,

I have converted this PR from WIP to an actual PR. The basic SQLAlchemy-based metadata ingestion is now working and has been tested via CLI. Additionally, I have confirmed that the UI part should function correctly, though I have not tested the ingestion via UI yet, as I am not familiar with Airflow. I aimed to keep the initial PR for the ingestion plugin minimal. I plan to create additional PRs, such as one for lineage, in the future once we have the basic plugin merged.

If you require a basic set of unit/integration tests, it would be helpful to have some guidance on what kind of tests you typically expect to be included.

I mostly oriented myself on this PR.

Describe your changes:

feat : Add support for exasol datasource

Exasol DB is a high-performance, in-memory, massively parallel processing (MPP) database specifically designed for advanced analytics and business intelligence applications.

Type of change:

  • New feature

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.
  • The issue properly describes why the new feature is needed, what's the goal, and how we are building it. Any discussion
    or decision-making process is reflected in the issue.
  • I have updated the documentation.
  • I have added tests around the new logic.

@Nicoretti Nicoretti marked this pull request as draft July 24, 2024 13:49
@github-actions GitHub Actions
Copy link
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@ulixius9
Copy link
Member

ulixius9 commented Jul 30, 2024 •
edited
Loading

Question: For the final PR, should the branch history be cleaned up so it can be rebased, or will it be squash merged anyway?

Hey @Nicoretti thanks for the pr, it will get squash merged so dont worry about the branch history

whenever your pr is ready we will add the safe to test label to get the tests kick in

pmbrull
pmbrull reviewed Jul 30, 2024
View reviewed changes
exasol-notes.md Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@pmbrull pmbrull added the safe to test Add this label to run secure Github workflows on PRs label Oct 29, 2024
pmbrull
pmbrull reviewed Oct 29, 2024
View reviewed changes
Copy link
Collaborator

@pmbrull pmbrull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Nicoretti,

Appreciate the time here, great PR!

Left a quick comment. LMK if you need help.

Thanks!

ingestion/setup.py Outdated Show resolved Hide resolved
ingestion/src/metadata/ingestion/source/database/exasol/connection.py Show resolved Hide resolved
ingestion/src/metadata/ingestion/source/database/exasol/connection.py Outdated Show resolved Hide resolved
@github-actions GitHub Actions
Copy link
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 64%
 64.11% (37994/59260) 40.75% (15159/37199) 42.76% (4541/10620)

@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Failed Quality Gate failed for 'open-metadata-ingestion'

Failed conditions
0.0% Coverage on New Code (required ≥ 20%)

See analysis details on SonarCloud

@sonarqubecloud SonarQubeCloud
Copy link

Quality Gate Passed Quality Gate passed for 'open-metadata-ui'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

pmbrull
pmbrull approved these changes Oct 30, 2024
View reviewed changes
Copy link
Collaborator

@pmbrull pmbrull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks great!

@pmbrull pmbrull merged commit 7ebc62d into open-metadata:main Oct 31, 2024
25 of 26 checks passed
@Nicoretti Nicoretti deleted the exasol-connector branch October 31, 2024 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pmbrull pmbrull pmbrull approved these changes

@ShaileshParmar11 ShaileshParmar11 Awaiting requested review from ShaileshParmar11

@karanh37 karanh37 Awaiting requested review from karanh37

@chirag-madlani chirag-madlani Awaiting requested review from chirag-madlani

@Sachin-chaurasiya Sachin-chaurasiya Awaiting requested review from Sachin-chaurasiya

Assignees
No one assigned
Labels
backend e2e:Integration Ingestion safe to test Add this label to run secure Github workflows on PRs UI UI specific issues
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@Nicoretti @ulixius9 @pmbrull @chirag-madlani